A common artifact typically present in statistical, predictive, and perceptual image compression systems is a reduction of color or brightness dynamic range. This is intentional, in most cases, since the quantization applied to reduce the dynamic range is not meant to be perceivable by most humans under normal viewing conditions. However, as channel transmission bandwidths become more and more constrained, and more and more compression of images and video is demanded, typical encoders—and accordingly their matched decoders—introduce several objectionable artifacts at the final viewing end under these constrained conditions.
In addition to blocking artifacts prevalent with previous generation encoding standards such as MPEG2, artifacts introduced by contemporaneous compression technologies such as H.264 or MPEG-4 Part 10, Advanced Video Coding (MPEG-4 AVC), and High Efficiency Video Coding (HEVC), have traded blocking artifacts for other artifacts such as blurring and color-banding when transmission capacity is under-provisioned. Color banding is one of the most noticeable of these artifacts because of spatial high-frequency color contours that are readily apparent in still images, and more apparent when the banding or contour edges are moving within an image sequence. In addition, encoding standards that frequently exhibit color banding, such as JPEG, are likely to continue to be prevalent because of their early success. These artifacts have several names known in the art; less formally, called “banding”, or “color contouring”. These artifacts are more broadly known as dynamic range reduction, since they are a result of digital quantization of a signal under analysis.
Alleviating such artifacts has many possible solutions, such as increasing the transmission bandwidth or increasing color space subsampling from YUV420 subsampling to those such as YUV444, but come at considerable costs. One solution would treat color banding artifacts as a post-decoder post-processing solution prior to rendering to a display device.
Attempts have been made to address the general problem of generating high-dynamic range images from low-dynamic range images, but they suffer from some critical disadvantages.
U.S. Patent Application Publication No. 2014/0079335 discloses a high-dynamic-range imaging system that analyses relative exposure levels by using motion analysis. However, this approach only works for video and moving imagery, and is not suitable for single images.
U.S. Patent Application Publication No. 2013/0107956 (hereinafter “the '956 publication”) discloses a method to generate high dynamic range images from low dynamic range images. However, the method disclosed in the '956 publication employs a predictive mapping that is closely coupled to the image encoder and decoder, which requires changes to an end-user device decoder. This renders the method disclosed in the '956 publication infeasible to employ for most mobile applications. Additionally, the method disclosed in the '956 publication requires high-dynamic range reference imagery such as YUV444, which is usually not available.
U.S. Patent Application Publication No. 2014/0177706 (hereinafter “the '706 publication”) discloses a method and system that provides super-resolution of quantized images and video. While the method disclosed in the '706 publication could be altered to restore dynamic range instead of spatial resolution, the method disclosed in the '706 publication is similarly closely coupled to the encoder and decoder processes, rendering the method disclosed in the '706 publication infeasible for mobile applications.
Other related art methods specifically attempt to solve the problem of banding and contouring of quantized images, such as found with aggressively compressed images or video, with a decoupled post-processing method or system. Typically such methods operate by selectively blurring parts of a given low-dynamic range image, and differ primarily by the method that masks which regions are and are not spatially blurred. Such filtering is a subset of the general dynamic range restoration problem.
U.S. Pat. No. 8,582,913 (hereinafter “the '913 patent”) discloses a post-processing controller to generate high-dynamic range images from low dynamic range images. While the controller as disclosed is amenable to real-time operations, it uses a brightness enhancement function tied to the angular spatial frequency represented by features in an image, which requires that the disclosed process have information about the display width and height, and the intended distance of the viewer from the display, limiting its application to professional applications and rendering the method disclosed in the '913 patent unsuitable for general consumer applications. Furthermore, the method disclosed in the '913 patent uses simple thresholds in its masking operation, meaning that it is prone to false positives such as smoothing areas with no banding but legitimate image detail, and false negatives such as not eliminating contouring artifacts where they should be treated.
International PCT Publication No. WO/2005/101309 (hereinafter “the '309 publication”) also discloses methods and systems for converting images from low dynamic range to high dynamic range. However, like several others, the methods disclosed in the '309 publication uses simple thresholds in its masking operation, meaning that it is not robust, and prone to false positives such as smoothing areas with no banding but legitimate image detail, and false negatives such as not eliminating contouring artifacts where they should be treated.
Such banding and contouring artifacts may be treated in the frequency domain by selective convolution, however the process of converting to and from the frequency domain renders selective convolution ineffective for processing full-resolution imagery on contemporaneous mobile consumer devices.
In addition, the foregoing methods are generally applied to a single purpose, namely the reduction of image banding and contouring artifacts due to limited dynamic range of digital sensors or compression-related quantization. The foregoing methods have no general applicability to other related problems in the art such as quantization errors of vertex location of 3D models, and the restoration of dynamic range of other D-dimensional datasets.